Influence of autistic traits and communication role on eye contact behavior during face-to-face interaction.

Eye contact is a central component in face-to-face interactions. It is important in structuring communicative exchanges and offers critical insights into others' interests and intentions. To better understand eye contact in face-to-face interactions, we applied a novel, non-intrusive deep-learning-based dual-camera system and investigated associations between eye contact and autistic traits as well as self-reported eye contact discomfort during a referential communication task, where participants and the experimenter had to guess, in turn, a word known by the other individual. Corroborating previous research, we found that participants' eye gaze and mutual eye contact were inversely related to autistic traits. In addition, our findings revealed different behaviors depending on the role in the dyad: listening and guessing were associated with increased eye contact compared with describing words. In the listening and guessing condition, only a subgroup who reported eye contact discomfort had a lower amount of eye gaze and eye contact. When describing words, higher autistic traits were associated with reduced eye gaze and eye contact. Our data indicate that eye contact is inversely associated with autistic traits when describing words, and that eye gaze is modulated by the communicative role in a conversation.

Why did Rubens add a parrot to Titian's The Fall of Man? A pictorial manipulation of joint attention.

Almost 400 years ago, Rubens copied Titian's The Fall of Man, albeit with important changes. Rubens altered Titian's original composition in numerous ways, including by changing the gaze directions of the depicted characters and adding a striking red parrot to the painting. Here, we quantify the impact of Rubens's choices on the viewer's gaze behavior. We displayed digital copies of Rubens's and Titian's artworks-as well as a version of Rubens's painting with the parrot digitally removed-on a computer screen while recording the eye movements produced by observers during free visual exploration of each image. To assess the effects of Rubens's changes to Titian's composition, we directly compared multiple gaze parameters across the different images. We found that participants gazed at Eve's face more frequently in Rubens's painting than in Titian's. In addition, gaze positions were more tightly focused for the former than for the latter, consistent with different allocations of viewer interest. We also investigated how gaze fixation on Eve's face affected the perceptual visibility of the parrot in Rubens's composition and how the parrot's presence versus its absence impacted gaze dynamics. Taken together, our results demonstrate that Rubens's critical deviations from Titian's painting have powerful effects on viewers' oculomotor behavior.

Impaired recognition of interactive intentions in adults with autism spectrum disorder not attributable to differences in visual attention or coordination via eye contact and joint attention.

Altered nonverbal communication patterns especially with regard to gaze interactions are commonly reported for persons with autism spectrum disorder (ASD). In this study we investigate and differentiate for the first time the interplay of attention allocation, the establishment of shared focus (eye contact and joint attention) and the recognition of intentions in gaze interactions in adults with ASD compared to control persons. Participants interacted via gaze with a virtual character (VC), who they believed was controlled by another person. Participants were instructed to ascertain whether their partner was trying to interact with them. In fact, the VC was fully algorithm-controlled and showed either interactive or non-interactive gaze behavior. Participants with ASD were specifically impaired in ascertaining whether their partner was trying to interact with them or not as compared to participants without ASD whereas neither the allocation of attention nor the ability to establish a shared focus were affected. Thus, perception and production of gaze cues seem preserved while the evaluation of gaze cues appeared to be impaired. An additional exploratory analysis suggests that especially the interpretation of contingencies between the interactants' actions are altered in ASD and should be investigated more closely.

Common structure of saccades and microsaccades in visual perception.

We obtain large amounts of external information through our eyes, a process often considered analogous to picture mapping onto a camera lens. However, our eyes are never as still as a camera lens, with saccades occurring between fixations and microsaccades occurring within a fixation. Although saccades are agreed to be functional for information sampling in visual perception, it remains unknown if microsaccades have a similar function when eye movement is restricted. Here, we demonstrated that saccades and microsaccades share common spatiotemporal structures in viewing visual objects. Twenty-seven adults viewed faces and houses in free-viewing and fixation-controlled conditions. Both saccades and microsaccades showed distinctive spatiotemporal patterns between face and house viewing that could be discriminated by pattern classifications. The classifications based on saccades and microsaccades could also be mutually generalized. Importantly, individuals who showed more distinctive saccadic patterns between faces and houses also showed more distinctive microsaccadic patterns. Moreover, saccades and microsaccades showed a higher structure similarity for face viewing than house viewing and a common orienting preference for the eye region over the mouth region. These findings suggested a common oculomotor program that is used to optimize information sampling during visual object perception.

Gaze patterns reflect the retrieval and selection of memories in a context-dependent object location retrieval task.

Selective retrieval of context-relevant memories is critical for animal survival. A behavioral index that captures its dynamic nature in real time is necessary to investigate this retrieval process. Here, we found a bias in eye gaze towards the locations previously associated with individual objects during retrieval. Participants learned two locations associated with each visual object and recalled one of them indicated by a contextual cue in the following days. Before the contextual cue presentation, participants often gazed at both locations associated with the given object on the background screen (look-at-both), and the frequency of look-at-both gaze pattern increased as learning progressed. Following the cue presentation, their gaze shifted toward the context-appropriate location. Interestingly, participants showed a higher accuracy of memory retrieval in trials where they gazed at both object-associated locations, implying functional advantage of the look-at-both gaze patterns. Our findings indicate that naturalistic eye movements reflect the dynamic process of memory retrieval and selection, highlighting the potential of eye gaze as an indicator for studying these cognitive processes.

Toward a theory of perspective perception in pictures.

This paper reviews projection models and their perception in realistic pictures, and proposes hypotheses for three-dimensional (3D) shape and space perception in pictures. In these hypotheses, eye fixations, and foveal vision play a central role. Many past theories and experimental studies focus solely on linear perspective. Yet, these theories fail to explain many important perceptual phenomena, including the effectiveness of nonlinear projections. Indeed, few classical paintings strictly obey linear perspective, nor do the best distortion-avoidance techniques for wide-angle computational photography. The hypotheses here employ a two-stage model for 3D human vision. When viewing a picture, the first stage perceives 3D shape for the current gaze. Each fixation has its own perspective projection, but, owing to the nature of foveal and peripheral vision, shape information is obtained primarily for a small region of the picture around the fixation. As a viewer moves their eyes, the second stage continually integrates some of the per-gaze information into an overall interpretation of a picture. The interpretation need not be geometrically stable or consistent over time. It is argued that this framework could explain many disparate pictorial phenomena, including different projection styles throughout art history and computational photography, while being consistent with the constraints of human 3D vision. The paper reviews open questions and suggests new studies to explore these hypotheses.

Temporal progression of pupil dilation and gaze behavior to emotion expressions in preschoolers with autism spectrum disorder.

Previous work has shown divergent pupil dilation (PD) and gaze behavior in individuals with autism spectrum disorder (ASD), which may relate to the development of social difficulties in early life. Here, we investigated temporal dynamics of both phenotypes during naturalistic videos of a person displaying facial emotion expressions in 61 autistic and 61 non-autistic preschoolers. PD was segmented into three serial time components derived from a principal component analysis. Growth curve analysis was applied to analyze changes in looking time on eye and mouth regions over time. Groups did not differ in PD time components. Growth curve analysis revealed initially shorter looking times on the eyes and longer looking times on the mouth in autistic versus non-autistic preschoolers. However, a reversion of this pattern was observed over time, suggesting a delayed compensatory increase in eye attention during prolonged viewing periods in autistic children. Positive and negative associations of PD components and gaze behavior over time indicated a dynamic temporal relationship during emotion viewing. Our findings emphasize the need to apply time-sensitive measures in ecologically valid research, which may index etiological mechanisms of social difficulties in ASD.

Attentional cueing: Gaze is harder to override than arrows.

Gaze is an important and potent social cue to direct others' attention towards specific locations. However, in many situations, directional symbols, like arrows, fulfill a similar purpose. Motivated by the overarching question how artificial systems can effectively communicate directional information, we conducted two cueing experiments. In both experiments, participants were asked to identify peripheral targets appearing on the screen and respond to them as quickly as possible by a button press. Prior to the appearance of the target, a cue was presented in the center of the screen. In Experiment 1, cues were either faces or arrows that gazed or pointed in one direction, but were non-predictive of the target location. Consistent with earlier studies, we found a reaction time benefit for the side the arrow or the gaze was directed to. Extending beyond earlier research, we found that this effect was indistinguishable between the vertical and the horizontal axis and between faces and arrows. In Experiment 2, we used 100% "counter-predictive" cues; that is, the target always occurred on the side opposite to the direction of gaze or arrow. With cues without inherent directional meaning (color), we controlled for general learning effects. Despite the close quantitative match between non-predictive gaze and non-predictive arrow cues observed in Experiment 1, the reaction-time benefit for counter-predictive arrows over neutral cues is more robust than the corresponding benefit for counter-predictive gaze. This suggests that-if matched for efficacy towards their inherent direction-gaze cues are harder to override or reinterpret than arrows. This difference can be of practical relevance, for example, when designing cues in the context of human-machine interaction.

Who says you are so sick? An investigation on individual susceptibility to cybersickness triggers using EEG, EGG and ECG.

In this research paper, we conducted a study to investigate the connection between three objective measures: Electrocardio-gram(EGG), Electrogastrogram (EGG), and Electroencephalogram (EEG), and individuals' susceptibility to cybersickness. Our primary objective was to identify which of these factors plays a central role in causing discomfort when experiencing rotations along three different axes: Roll, Pitch, and Yaw. This study involved 35 participants who were tasked with destroying asteroids using their eye gaze while undergoing passive rotations in four separate sessions. The results, when combined with subjective measurements (specifically, Fast motion sickness questionnaire (FMS) and Simulator sickness questionnaire (SSQ) score), demonstrated that EGG measurements were superior in detecting symptoms associated with nausea. As for ECG measurements, our observations did reveal significant changes in Heart Rate Variability (HRV) parameters. However, we caution against relying solely on ECG as a dependable indicator for assessing the extent of cybersickness. Most notably, EEG signals emerged as a crucial resource for discerning individual differences related to these rotational axes. Our findings were significant not only in the context of periodic activities but also underscored the potential of aperiodic activities in detecting the severity of cybersickness and an individual's susceptibility to rotational triggers.

Motor "laziness" constrains fixation selection in real-world tasks.

Humans coordinate their eye, head, and body movements to gather information from a dynamic environment while maximizing reward and minimizing biomechanical and energetic costs. However, such natural behavior is not possible in traditional experiments employing head/body restraints and artificial, static stimuli. Therefore, it is unclear to what extent mechanisms of fixation selection discovered in lab studies, such as inhibition-of-return (IOR), influence everyday behavior. To address this gap, participants performed nine real-world tasks, including driving, visually searching for an item, and building a Lego set, while wearing a mobile eye tracker (169 recordings; 26.6 h). Surprisingly, in all tasks, participants most often returned to what they just viewed and saccade latencies were shorter preceding return than forward saccades, i.e., consistent with facilitation, rather than inhibition, of return. We hypothesize that conservation of eye and head motor effort ("laziness") contributes. Correspondingly, we observed center biases in fixation position and duration relative to the head's orientation. A model that generates scanpaths by randomly sampling these distributions reproduced all return phenomena we observed, including distinct 3-fixation sequences for forward versus return saccades. After controlling for orbital eccentricity, one task (building a Lego set) showed evidence for IOR. This, along with small discrepancies between model and data, indicates that the brain balances minimization of motor costs with maximization of rewards (e.g., accomplished by IOR and other mechanisms) and that the optimal balance varies according to task demands. Supporting this account, the orbital range of motion used in each task traded off lawfully with fixation duration.

Individual differences in human gaze behavior generalize from faces to objects.

Individuals differ in where they fixate on a face, with some looking closer to the eyes while others prefer the mouth region. These individual biases are highly robust, generalize from the lab to the outside world, and have been associated with social cognition and associated disorders. However, it is unclear, whether these biases are specific to faces or influenced by domain-general mechanisms of vision. Here, we juxtaposed these hypotheses by testing whether individual face fixation biases generalize to inanimate objects. We analyzed >1.8 million fixations toward faces and objects in complex natural scenes from 405 participants tested in multiple labs. Consistent interindividual differences in fixation positions were highly inter-correlated across faces and objects in all samples. Observers who fixated closer to the eye region also fixated higher on inanimate objects and vice versa. Furthermore, the inter-individual spread of fixation positions scaled with target size in precisely the same, non-linear manner for faces and objects. These findings contradict a purely domain-specific account of individual face gaze. Instead, they suggest significant domain-general contributions to the individual way we look at faces, a finding with potential relevance for basic vision, face perception, social cognition, and associated clinical conditions.

Age-Differential Role of Gaze Reinstatement in Recognition Memory for Negative Visual Stimuli.

OBJECTIVES: Although research has shown that the replay of encoding-specific gaze patterns during retrieval, known as gaze reinstatement, facilitates memory retrieval, little is known about whether it differentially associates with the negativity preference in memory (defined as enhanced memory for negative stimuli relative to neutral stimuli in this study) among younger and older adults. The present study aims to address this research gap. METHODS: A total of 33 older adults (16 women; aged 58-69 years, M = 63.48, SD = 2.98) and 36 younger adults (10 women; aged 18-26 years, M = 20.39, SD = 1.57) completed a remember/know recognition memory task involving negative and neutral pictures. Their eye movements were tracked during both the memory encoding and retrieval phases. RESULTS: Younger and older adults had better memory for negative than neutral pictures. Older adults exhibited significantly stronger gaze reinstatement for negative than neutral stimuli, while this difference was nonsignificant in younger adults. Moreover, gaze reinstatement is positively linked to memory performance in both age groups. DISCUSSION: The results suggest that gaze reinstatement may play age-differential roles in the negativity preference of memory. Negative valence may enhance gaze reinstatement, which improves subsequent recognition memory, particularly among older adults. The finding contributes to a better understanding of the mechanisms underlying the negative preference for memory in different age groups.

Eye-Hand Typing: Eye Gaze Assisted Finger Typing via Bayesian Processes in AR.

Nowadays, AR HMDs are widely used in scenarios such as intelligent manufacturing and digital factories. In a factory environment, fast and accurate text input is crucial for operators' efficiency and task completion quality. However, the traditional AR keyboard may not meet this requirement, and the noisy environment is unsuitable for voice input. In this article, we introduce Eye-Hand Typing, an intelligent AR keyboard. We leverage the speed advantage of eye gaze and use a Bayesian process based on the information of gaze points to infer users' text input intentions. We improve the underlying keyboard algorithm without changing user input habits, thereby improving factory users' text input speed and accuracy. In real-time applications, when the user's gaze point is on the keyboard, the Bayesian process can predict the most likely characters, vocabulary, or commands that the user will input based on the position and duration of the gaze point and input history. The system can enlarge and highlight recommended text input options based on the predicted results, thereby improving user input efficiency. A user study showed that compared with the current HoloLens 2 system keyboard, Eye-Hand Typing could reduce input error rates by 28.31 % and improve text input speed by 14.5%. It also outperformed a gaze-only technique, being 43.05% more accurate and 39.55% faster. And it was no significant compromise in eye fatigue. Users also showed positive preferences.

[The effect of pleoptic treatment on the stability of visual fixation in children and adolescents with amblyopia of various degrees]. / Vliyanie pleopticheskogo lecheniya na stabil'nost' zritel'noi fiksatsii u detei i podrostkov s ambliopiei po dannym mikroperimetrii.

PURPOSE: This study assesses the effect of pleoptic treatment on the stability of visual fixation in children and adolescents with amblyopia of various degrees. MATERIAL AND METHODS: The effect of the treatment in 35 children (35 eyes) with amblyopia of various degrees was evaluated using the MP-1 Microperimeter. The stability was determined by two criteria - number of fixation point hits in the 2° zone and the width of the field of fixation. Children underwent a standard course of treatment. RESULTS: There was an improvement in the stability of visual fixation after treatment. The number of fixation point hits in the 2° zone increased from 44.8±4.1% to 52.6±3.8%. At the same time, the width of the field of fixation after treatment decreased from 6.5±0.7° to 5.2±0.5°. Visual acuity with correction in children with amblyopia correlates with the number of fixation point hits in the 2° zone (direct high correlation "+0.7") and width of the field of fixation (inverse high correlation "-0.7") both before and after the treatment. CONCLUSION: Stabilization of visual fixation was observed in patients with amblyopia of varying degrees after the pleoptic treatment.

Coordination of gaze and action during high-speed steering and obstacle avoidance.

When humans navigate through complex environments, they coordinate gaze and steering to sample the visual information needed to guide movement. Gaze and steering behavior have been extensively studied in the context of automobile driving along a winding road, leading to accounts of movement along well-defined paths over flat, obstacle-free surfaces. However, humans are also capable of visually guiding self-motion in environments that are cluttered with obstacles and lack an explicit path. An extreme example of such behavior occurs during first-person view drone racing, in which pilots maneuver at high speeds through a dense forest. In this study, we explored the gaze and steering behavior of skilled drone pilots. Subjects guided a simulated quadcopter along a racecourse embedded within a custom-designed forest-like virtual environment. The environment was viewed through a head-mounted display equipped with an eye tracker to record gaze behavior. In two experiments, subjects performed the task in multiple conditions that varied in terms of the presence of obstacles (trees), waypoints (hoops to fly through), and a path to follow. Subjects often looked in the general direction of things that they wanted to steer toward, but gaze fell on nearby objects and surfaces more often than on the actual path or hoops. Nevertheless, subjects were able to perform the task successfully, steering at high speeds while remaining on the path, passing through hoops, and avoiding collisions. In conditions that contained hoops, subjects adapted how they approached the most immediate hoop in anticipation of the position of the subsequent hoop. Taken together, these findings challenge existing models of steering that assume that steering is tightly coupled to where actors look. We consider the study's broader implications as well as limitations, including the focus on a small sample of highly skilled subjects and inherent noise in measurement of gaze direction.

Visual Feature Tuning Properties of Short-Latency Stimulus-Driven Ocular Position Drift Responses during Gaze Fixation.

Ocular position drifts during gaze fixation are significantly less well understood than microsaccades. We recently identified a short-latency ocular position drift response, of ∼1 min arc amplitude, that is triggered within <100 ms by visual onsets. This systematic eye movement response is feature-tuned and seems to be coordinated with a simultaneous resetting of the saccadic system by visual stimuli. However, much remains to be learned about the drift response, especially for designing better-informed neurophysiological experiments unraveling its mechanistic substrates. Here we systematically tested multiple new feature tuning properties of drift responses. Using highly precise eye tracking in three male rhesus macaque monkeys, we found that drift responses still occur for tiny foveal visual stimuli. Moreover, the responses exhibit size tuning, scaling their amplitude (both up and down) as a function of stimulus size, and they also possess a monotonically increasing contrast sensitivity curve. Importantly, short-latency drift responses still occur for small peripheral visual targets, which additionally introduce spatially directed modulations in drift trajectories toward the appearing peripheral stimuli. Drift responses also remain predominantly upward even for stimuli exclusively located in the lower visual field and even when starting gaze position is upward. When we checked the timing of drift responses, we found it was better synchronized to stimulus-induced saccadic inhibition than to stimulus onset. These results, along with a suppression of drift response amplitudes by peristimulus saccades, suggest that drift responses reflect the rapid impacts of short-latency and feature-tuned visual neural activity on final oculomotor control circuitry in the brain.

Similar social attention, physiological arousal, and familiarity effect in autistic and neurotypical children: A real-life recreational eye-tracking paradigm.

Social attention is reported to be crucial for the development of social skills, and, according to the social cognitive developmental theory, is fostered by social interactions. Autism is of central importance to the study of social attention, as autism is characterized by atypical social interactions and low social attention, both linked according to the social motivation theory to diminished social interest. Much evidence for positing low social interest in autism comes from eye-tracking studies, which, however, lack ecological validity. Our study documents social attention and physiological arousal, within close to real-life settings, in autistic children, as well as in their neurotypical peers, matched on gender and mental or chronological age. To explore the potential influence of partner familiarity or of the conversational topic, children gaze and electrodermal activity were recorded while they engaged in watercolor activities with, first a familiar and, next, an unfamiliar adult experimenter, who both introduced various topics. Autistic and neurotypical children exhibited comparable attention to their partners' eyes. Notably, across all groups, heightened visual attention was directed to familiar rather than unfamiliar partners. Moreover, parallel arousal patterns emerged, with all children displaying increased skin conductance responses during more engaging topics and when looking at their interactional partner's eyes. These findings underscore the task- and context-dependent nature of social attention and highlight the role of familiarity in an ecologically valid context. The absence of group differences challenges the universality of the social cognitive developmental theory and questions the scope of the social motivation theory of autism. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

Vision and microsaccades: Time to pay attention!

Visual perception of exceedingly small and highly detailed spatial regions depends on coordinated patterns of small shifts of the line of sight ('microsaccades') aided by pre-saccadic shifts of spatial attention directed precisely to the intended target of the saccade.

Parafoveal and foveal N400 effects in natural reading: A timeline of semantic processing from fixation-related potentials.

The depth at which parafoveal words are processed during reading is an ongoing topic of debate. Recent studies using RSVP-with-flanker paradigms have shown that implausible words within sentences elicit an N400 component while they are still in parafoveal vision, suggesting that the semantics of parafoveal words can be accessed to rapidly update the sentence representation. To study this effect in natural reading, we combined the coregistration of eye movements and EEG with the deconvolution modeling of fixation-related potentials (FRPs) to test whether semantic plausibility is processed parafoveally during Chinese sentence reading. For one target word per sentence, both its parafoveal and foveal plausibility were orthogonally manipulated using the boundary paradigm. Consistent with previous eye movement studies, we observed a delayed effect of parafoveal plausibility on fixation durations that only emerged on the foveal word. Crucially, in FRPs aligned to the pretarget fixation, a clear N400 effect emerged already based on parafoveal plausibility, with more negative voltages for implausible previews. Once participants fixated the target, we again observed an N400 effect of foveal plausibility. Interestingly, this foveal N400 was absent whenever the preview had been implausible, indicating that when a word's (im)plausibility is already processed in parafoveal vision, this information is not revised anymore upon direct fixation. Implausible words also elicited a late positive component (LPC), but exclusively when in foveal vision. Our results not only provide convergent neural and behavioral evidence for the parafoveal uptake of semantic information, but also indicate different contributions of parafoveal versus foveal information toward higher level sentence processing.

Shedding light on ai in radiology: A systematic review and taxonomy of eye gaze-driven interpretability in deep learning.

X-ray imaging plays a crucial role in diagnostic medicine. Yet, a significant portion of the global population lacks access to this essential technology due to a shortage of trained radiologists. Eye-tracking data and deep learning models can enhance X-ray analysis by mapping expert focus areas, guiding automated anomaly detection, optimizing workflow efficiency, and bolstering training methods for novice radiologists. However, the literature shows contradictory results regarding the usefulness of eye-tracking data in deep-learning architectures for abnormality detection. We argue that these discrepancies between studies in the literature are due to (a) the way eye-tracking data is (or is not) processed, (b) the types of deep learning architectures chosen, and (c) the type of application that these architectures will have. We conducted a systematic literature review using PRISMA to address these contradicting results. We analyzed 60 studies that incorporated eye-tracking data in a deep-learning approach for different application goals in radiology. We performed a comparative analysis to understand if eye gaze data contains feature maps that can be useful under a deep learning approach and whether they can promote more interpretable predictions. To the best of our knowledge, this is the first survey in the area that performs a thorough investigation of eye gaze data processing techniques and their impacts in different deep learning architectures for applications such as error detection, classification, object detection, expertise level analysis, fatigue estimation and human attention prediction in medical imaging data. Our analysis resulted in two main contributions: (1) taxonomy that first divides the literature by task, enabling us to analyze the value eye movement can bring for each case and build guidelines regarding architectures and gaze processing techniques adequate for each application, and (2) an overall analysis of how eye gaze data can promote explainability in radiology.